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(54) Title: MOTION MOUSE SYSTEM 





(57) Abstract: An apparatus for inputting three-dimensional movements and status information includes a motion mouse having at 
^ least one indicator showing the status information, which is arranged on a first surface of the motion mouse, and at least one control 
^» switch for controlling a status of the indicator, which is arranged on a second surface of the motion mouse, a camera for capturing an 
£2 image of the motion mouse and an image analyzer for analyzing the image of the motion mouse by using the lens formula to recognize 

the three-dimensional movements and the status information of the motion mouse. The indicators may have different color, size or 
Q shape from each other to effectively represent a rotational movement of the motion mouse. The apparatus inputs three-dimensional 
£^ movements of the motion mouse in a simple and cost-effective manner without transmitting/receiving any data between a computer 
^ and a pointing device through wire or wireless connection. 
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MOTION MOUSE SYSTEM 

Technical Field 

5 The present invention relates to an apparatus and a 

method for inputting three-dimensional coordinates and 
status information of a pointing device into an information 
device such as a computer; and, more particularly, to an 
apparatus and a .method for analyzing an image obtained by 
10 capturing a movement of a pointing device in a three- 
dimensional space and recognizing changes of three- 
dimensional coordinates and status information of the 
pointing device. 

15 Background Art 

Generally, as a pointing device for positioning a 
cursor on a display screen of a computer, a device called 
"mouse" has been used. Such a pointing device detects its 

20 relative displacement over a plane by employing an optical 
method or a mechanical method for detecting a movement of a 
ball installed inside the pointing device. The detected 
displacement is converted into a displacement of the cursor 
on the display screen. However, since the mouse is designed 

25 to move along a two-dimensional plane, it has a limitation 
that three-dimensional movements of the mouse cannot be 



WO 03/079179 



PCT7KR03/00524 



transferred to the computer. 

Meanwhile, as users of personal computers or CAD 
(computer aided design) systems require a device for 
inputting positions and shapes of various three-dimensional 
5 objects, a variety of pointing devices for inputting three- 
dimensional coordinates have been proposed. Such three- 
dimensional pointing devices may have a function to measure 
its movement along the z-axis in addition to its two- 
dimensional movement over the x-y coordinate system. 

10 Further, the three-dimensional pointing devices may employ 
an accelerometer for detecting its rotating movement along 
the x, y and z axes. 

However, the prior art pointing device may cause its 
user inconvenience since the pointing device may need a 

15 supplementary device such as a supporting board on which the 
pointing device moves. Further, the prior art three- 
dimensional pointing device has to be located within a short 
distance from a computer when they are wire-connected to 
each other. If the pointing device is wireless-connected to 

20 the computer through, e.g., IR (infrared) light, the 
limitation of distance between the pointing device and the 
computer may be overcome. Even in this case, however, if an 
IR light emitting direction of one side, e.g., the pointing 
device, does not coincide with an IR light reception 

25 direction of the other side, e.g., the computer, data 
communications between the both sides may be interrupted. 
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Disclosure of Invention 

It is, therefore, an object of the present invention 
to provide an apparatus and a method for analyzing an image 
5 obtained by capturing a movement of a pointing device in a 
three-dimensional space in front of a computer and 
recognizing changes of three-dimensional coordinates and 
status information of the pointing device. 

In accordance with one aspect of the present invention, 

10 there is provided an apparatus for inputting three- 
dimensional coordinates and status information of a motion 
mouse, including: the motion mouse including more than one 
indicator, which are arranged on a first surface of the 
motion mouse, for representing the status information and 

15 more than one control switch, which are arranged on a second 
surface of the motion mouse, for controlling the status 
information represented by the indicators; a camera for 
capturing images of the indicators arranged on the first 
surface; and an image analyzer for analyzing the captured 

20 images and recognizing three-dimensional coordinates and 
status information of the motion mouse. 

In accordance with another aspect of the present 
invention, there is provided a method for inputting three- 
dimensional coordinates and status information of a motion 

25 mouse including more than one indicator, which are arranged 
on a first surface of the motion mouse, for representing the 
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status information and more than one control switch, which 
are arranged on a second surface of the motion mouse, for 
controlling the status information represented by the 
indicators, the method including the steps of: capturing 
5 images of the indicators; extracting areas of the indicators 
by using information on colors, sizes and/or shapes of the 
indicators or locations of the indicators on the first 
surface; determining center coordinates of the areas of the 
indicators; and determining changes of three-dimensional 
10 positions, an amount of rotation and status information of 
the motion mouse by using the determined center coordinates. 

Brief Description of Drawings 

15 The above and other objects and features of the 

present invention will become apparent from the following 

description of preferred embodiments given in conjunction 

with the accompanying drawings, in which: 

Fig. 1 sets forth an apparatus for inputting three- 
20 dimensional coordinates and status information of a motion 

mouse , in accordance with a preferred embodiment of the 

present invention; 

Fig. 2 depicts a motion mouse in accordance with a 

preferred embodiment of the present invention; 
25 Fig. 3 illustrates a rotation of a front surface of a 

motion mouse in accordance with a preferred embodiment of 
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the present invention, the front surface having two 
illuminators thereon; 

Fig, 4 provides a flowchart showing a method for 
inputting three-dimensional coordinates and status 
5 information of a motion mouse in accordance with a preferred 
embodiment of the present invention; 

Fig. 5 presents a diagram of a lens and an object for 
describing the concept of the lens formula; and 

Fig. 6 charts a rotation of a front surface of a 
10 motion mouse in accordance with a preferred embodiment of 
the present invention and changes of locations of 
illuminators attached on the front surface in accordance 
with the rotation of the front surface. 

15 Best Mode for Carrying Out the Invention 

Referring to Fig. 1, there is provided a configuration 
of an apparatus for inputting three-dimensional coordinates 
and status information of a motion mouse in accordance with 

20 a preferred embodiment of the present invention. As shown 
in Fig. 1, the apparatus includes a computer 110 , a camera 
120 for capturing an image, which is connected to the 
computer 110, a pointing device (hereinafter referred to as 
"motion mouse") for representing three-dimensional positions 

25 and status information. A user of the computer 110 
represents three-dimensional positions and status 
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information by moving the motion mouse 200 in a three- 
dimensional space in front of the camera 120. The camera 
120 captures images of the movements of the motion mouse 200, 
such that an analysis software executed in the computer 110 
5 analyzes the images captured by the camera 120 to recognize 
the three-dimensional positions and the status information 
of the motion mouse 200. 

Herein, since the motion mouse 200 does not have to 
transceive any information through a wired or wireless 

10 connection, the motion mouse 200 can move freely in a three- 
dimensional space in front of the camera 120. For instance, 
the motion mouse 200 may rotate or move in left/right and 
up/down directions or any combination thereof. The 
rotational movement of the motion mouse 200 may be 

15 decomposed into several rotational components on x, y and z 
axes (the z axis represents an axis passing through a center 
of a lens, i.e., angle of view, of the camera 120.) 

Fig. 2 illustrates a configuration of the motion mouse 
200 in accordance with a preferred embodiment of the present 

20 invention. The motion mouse 200 includes a plurality of 
illuminators 211 to 213 and a plurality of switches 221 to 
223 for controlling ON/OFF status of the illuminators 211 to 
213. The switches 221 to 223 respectively control 
flickering of the illuminators 211 to 213, so that a 

25 combination of the ON/OFF status of the illuminators 211 to 
213 represents status information such as 
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selection/cancellation of a specific function, scrolling of 
a computer screen and three-dimensional movements of the 
motion mouse 200. 

Although the illuminators 211 to 213 have been 
5 described to be arranged in a line on the front surface of 
the motion mouse 200 in Fig. 2, the illuminators 211 to 213 
may be arranged asymmetrically with respect to a center of 
the front surface of the motion mouse 200. When the 
illuminators 211 to 213 are arranged asymmetrically with 

10 respect to the center of the front surface of the motion 
mouse 200, the motion mouse 200 can represent any rotational 
movements. Further, as shown in Fig. 2, even when the 
illuminators 211 to 213 are arranged symmetrically with 
respect to the center of the front surface of the motion 

15 mouse 200, the illuminators 211 to 213 may be configured to 
have different sizes, shapes and/or colors. The number of 
the illuminators may be varied as long as the motion mouse 
can represent various three-dimensional movements. 

Fig. 3 charts an exemplary image of a front surface of 

20 the motion mouse 200, which is captured by the camera 120. 
In Fig. 3, the motion mouse 200 employs two illuminators LI 
and L2, each of which has a different size. In this case, 
although the motion mouse 200 has only two illuminators LI 
and L2, the motion mouse 200 can represent various 

25 rotational movements. 

In general, the illuminators 211 to 213 may be 
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designed to emit visible rays. Alternatively, in order to 
perform more rapidly and precisely analysis of an image 
captured by the camera 120, the illuminators 211 to 213 may 
be implemented to irradiate infrared rays. In this case, an 
5 infrared filter may be installed in front of the camera 120 
to filter the infrared rays emitted by the illuminators 211 
to 213. The illuminators 211 to 213 need a power supply, 
which is installed inside the motion mouse 200 or the 
computer 110, to produce the rays. However, the 

10 illuminators 211 to 213 may be implemented by using 
fluorescent materials, which eliminate any needs for the 
power supply. In order to easily distinguish one of the 
illuminators from one another, the illuminators 211 to 213 
may have different sizes, shapes and/or colors (or 

15 frequencies) . Further, there may be installed shutters in 
front of the illuminators 211 to 213 to replace the 
flickering (i.e., ON/OFF) functions of the illuminators 211 
to 213. In this case, the control switches 221 to 223 are 
configured to control the opening and shutting movements of 

20 the shutters. 

In the meantime, analysis software is executed inside 
the computer 110 for analyzing images of the motion mouse 
200, which are captured by the camera 120. In the following, 
a method for analyzing the images of the motion mouse in 

25 accordance with a preferred embodiment of the present 
invention will be described in detail with reference to Fig. 
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4. 

First, the camera 120 captures the movements of the 
motion mouse 200 and transfers the captured images of the 
motion mouse 200 to the computer 110 (step 402) . Then, the 
5 analysis software, which is executed in the computer 110, 
performs analysis of the captured images. 

The analysis software extracts areas of the 
illuminators in the images of the motion mouse 200 (step 
404). The analysis software may preprocess the images to 

10 easily extract the areas of the illuminators from the images. 
For example, a threshold value may be predetermined to 
convert pixel levels of the images into binary values, i.e., 
0 and 1, such that, when a pixel level of the images is 
larger than or equal to the threshold value, a high level, 

15 e.g., 1, is assigned to the pixel. Further, when the pixel 
level is smaller than the threshold value, a low level, e.g., 
0, is assigned to the pixel. Meanwhile, the computer 110 
stores information on locations, colors, sizes and/or shapes 
of the illuminators arranged on the front surface of the 

20 motion mouse 200, which is utilized by the analysis software 
to extract more precisely the areas of the illuminators. 

Thereafter, center coordinates of the extracted areas 
of the illuminators are calculated (step 406). The center 
coordinates of the extracted areas may be set to, e.g., 

25 centroids of the areas. 

The analysis software then reckons amounts of 
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forward/backward, right/left, up/down movements or rotations 
of the motion mouse 200 based on the center coordinates of 
the illuminators and the distances between the center 
coordinates (step 410) . Herein, the analysis software 
utilizes the lens formula in calculating changes in 
positions of the motion mouse 200 on the z axis. For 
instance, as shown in Fig. 5 f if it is assumed that a 
distance between an object 0 and a lens 510, a distance 
between an image I and the lens 510 and a focal distance of 
the lens 510 are b, a. and f, respectively, a mathematical 
expression ("the lens formula") as shown in Equation (1) can 
be formulated. 



In this case, a magnification m of the lens 510 can be 
determined as expressed in Equation (2) . 



Therefore, the analysis software determines the 
magnification m by measuring a distance (i.e., the size H of 
the object) between the illuminators of the motion mouse 200 
and a pixel distance (i.e., the size h of the image) between 




Equation (1) 



b H 



Equation (2) 



~ a h 
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the center coordinates of the illuminators. Further, the 
analysis software estimates the distance b by using the 
fixed distance a in accordance with Equation (2) . 
Accordingly, by measuring changes of the pixel distance 
5 (i.e., the. size h of the image) between the center 
coordinates of the illuminators, changes in the positions of 
the illuminators on the z axis can be calculated. 

Meanwhile, the amounts of rotations of the motion 
mouse 200 on the x, y and z axes may be calculated by 

10 measuring changes in the positions of the illuminators and 
the distances between the positions. For instance, as 
illustrated in Fig. 6, wherein four illuminators LI to L4 
having different shapes or colors are arranged crosswise 
with respect to a center C on the front surface of the 

15 motion mouse 200, a coordinate {Ax, Ay) of the center C on 
the x-y plane with respect to the z axis (0) and the amount 
(A0) of rotations of the illuminators LI to L4 may be 
estimated. Further, by measuring changes in a ratio of a 
distance between the illuminators LI and L3 and that between 

20 the illuminators L2 and L4, the amount of rotation of the 
motion mouse 200 with respect to the x axis or the y axis 
can be determined. 

The analysis software recognizes ON/OFF status or 
brightness of the illuminators while calculating changes of 

25 the positions of the illuminators (step 408) . And then, the 
ON/OFF status or brightness of the illuminators is converted 
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into information on selection/cancellation of specific 
functions or scrolling of a display screen. 

The determined three-dimensional positions and status 
information of the motion mouse 200 are then stored in a 
5 memory of the computer 110 to be provided for an application 
program (step 412) . The application program may be a 
browser program for navigating in a three-dimensional space 
composed by using three-dimensional graphics technology or a 
CAD program for editing three-dimensional objects. Although 
10 the method of the present invention has been described to be 
executed by software installed in the computer 110, a part 
or all of the steps of the method may be executed by an ad 
hoc hardware. 

As described above, the method and apparatus for 
15 inputting three-dimensional coordinates and status 
information of a motion mouse in accordance with the present 
invention transfers changes of the three-dimensional 
coordinates and status information of the motion mouse 
without performing wireless or wired data transmission 
20 between the motion mouse and the computer. Therefore, the 
method and apparatus of the present invention have an 
advantage that a user can input three-dimensional movements 
of a pointing device more intuitively. These aspects of the 
present invention contribute to convenience and portability 
25 of a pointing device. 

While the invention has been shown and described with 
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by those skilled in the art that various changes and 
modifications may be made without departing from the spirit 
and scope of the invention as defined in the following 
5 claims. 
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CLAIMS 

1. An apparatus for inputting three-dimensional 
coordinates and status information of a motion mouse, 

5 comprising: 

the motion mouse including more than one indicator, 
which are arranged on a first surface of the motion mouse, 
for representing the status information and more than one 
control switch, which are arranged on a second surface of 
10 the motion mouse, for controlling the status information 
represented by the indicators; 

a camera for capturing images of the indicators 
arranged on the first surface; and 

an image analyzer for analyzing the captured images 
15 and recognizing three-dimensional coordinates and status 
information of the motion mouse. 

2. The apparatus of claim 1, wherein the indicators are 
illuminators, which are switchable ON or OFF, and each of 

20 the control switches controls the ON/OFF of the 
corresponding illuminator. 

3. The apparatus of claim 2, wherein the illuminators 
emit infrared lights and the camera includes a filter for 

25 filtering the infrared lights. 
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4. The apparatus of claim 2, wherein the illuminators 
have different colors, sizes and/or shapes. 

5. The apparatus of claim 2, wherein the illuminators are 
5 arranged asymmetrically with respect to a center of the 

first surface. 

6. The apparatus of claim 1, wherein the motion mouse 
further includes shutters for opening and shutting the 

10 indicators, wherein the indicators have different colors, 
sizes and/or shapes and the control switches control the 
opening and shutting of the shutters. 

7. The apparatus of claim 6, wherein the illuminators are 
15 arranged asymmetrically with respect to a center of the 

first surface. 

8. The apparatus of claim 1, wherein the image analyzer 
performs the steps of: 

20 determining center coordinates of the indicators; 

calculating pixel distances between the center 
coordinates; and 

calculating changes in three-dimensional positions and 
an amount of rotation of the indicators by using the pixel 
25 distances and the lens formula. 
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9. The apparatus of claim 8, wherein the image analyzer 
analyzes the captured images to determine ON/OFF status, 
colors, sizes and/or shapes of the indicators, thereby 
determining the status information of the motion mouse. 

5 

10. A method for inputting three-dimensional coordinates 
and status information of a motion mouse including more than 
one indicator, which are arranged on a first surface of the 
motion mouse, for representing the status information and 

i0 more than one control switch, which are arranged on a second 
surface of the motion mouse, for controlling the status 
information represented by the indicators, the method 
comprising the steps of: 

capturing images of the indicators; 

15 extracting areas of the indicators by using 

information on colors, sizes and/or shapes of the indicators 
or locations of the indicators on the first surface; 

determining center coordinates of the areas of the 
indicators; and 

20 determining changes of three-dimensional positions, an 

amount of rotation and status information of the motion 
mouse by using the determined center coordinates. 

11. The method of claim 10, wherein the step of 
25 determining changes of the three-dimensional positions 

includes the steps of: 
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calculating pixel distances between the center 
coordinates; and 

calculating changes in three-dimensional positions and 
an amount of rotation of the indicators by using the pixel 
5 distances and the lens formula. 

12. The method of claim 11, wherein the step of 
determining changes in the three-dimensional positions 
further includes a step of analyzing the captured images to 
10 determine ON/OFF status, colors, sizes and/or shapes of the 
indicators, thereby determining the status information of 
the motion mouse. 
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AMENDED CLAIMS 
Received by the International bureau on 19 August 2003 (19.08.03); 
original claims 1 and 6 have been replaced by the amended claims. Claim 4 has been cancelled. 

1. An apparatus for inputting three-dimensional 
coordinates and status information of a motion mouse, 

5 comprising: 

the motion mouse including more than one indicator, 
which are arranged on a first surface of the motion mouse, 
for representing the status information and more than one 
control switch, which are arranged on a second surface of 
10 the motion mouse, for controlling the status information 
represented by the indicators, wherein the indicators have 
different colors, sizes and/or shapes; 

a camera for capturing images of the indicators 
arranged on the first surface; and 
15 an image analyzer for analyzing the captured images 

and recognizing three-dimensional coordinates and status 
information of the motion mouse. 

2. The apparatus of claim 1, wherein the indicators are 
20 illuminators, which are switchable ON or OFF, and each of 

the control switches controls the ON/OFF of the 
corresponding illuminator, 

3. The apparatus of claim 2, wherein the illuminators 
25 emit infrared lights and the camera includes a filter for 

filtering the infrared lights. 
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4. (Cancelled) 

5. The apparatus of claim 2, wherein the illuminators are 
5 arranged asymmetrically with respect to a center of the 

first surface, 

6. The apparatus of claim 1, wherein the motion mouse 
further includes shutters for opening and shutting the 

10 indicators, wherein the control switches control the opening 
and shutting of the shutters. 

7. The apparatus of claim 6, wherein the illuminators are 
arranged asymmetrically with respect to a center of the 

15 first surface. 

8. The apparatus of claim 1, wherein the image analyzer 
performs the steps of: 

determining center coordinates of the indicators; 
20 calculating pixel distances between the center 

coordinates; and 

calculating changes in three-dimensional positions and 
an amount of rotation of the indicators by using the pixel 
distances and the lens formula. 

25 

9. The apparatus of claim 8, wherein the image analyzer 
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analyzes the captured images to determine ON/OFF status, 
colors, sizes and/or shapes of the indicators, thereby 
determining the status information of the motion mouse. 

5 10. A method for inputting three-dimensional coordinates 
and status information of a motion mouse including more than 
one indicator, which are arranged on a first surface of the 
motion mouse, for representing the status information and 
more than one control switch, which are arranged on a second 
10 surface of the motion mouse, for controlling the status 
information represented by the indicators, the method 
comprising the steps of: 

capturing images of the indicators; 

extracting areas of the indicators by using 
15 information on colors, sizes and/or shapes of the indicators 
or locations of the indicators on the first surface; 

determining center coordinates of the areas of the 
indicators ; and 

determining changes of three-dimensional positions, an 
20 amount of rotation and status information of the motion 
mouse by using the determined center coordinates. 

11. The method of claim 10, wherein the step of 
determining changes of the three-dimensional positions 
25 includes the steps of: 

calculating pixel distances between the center 
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coordinates; and 

calculating changes in three-dimensional positions and 
an amount of rotation of the indicators by using the pixel 
distances and the lens formula. 

5 

12. The method of claim 11, wherein the step of 
determining changes in the three-dimensional positions 
further includes a step of analyzing the captured images to 
determine ON/OFF status, colors, sizes and/or shapes of the 
10 indicators, thereby determining the status information of 
the motion mouse. 
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